Demand-based Document Dissemination for the World-Wide Web

We analyzed the logs of our departmental HTTP server http://cs-www.bu.edu as well as the logs of the more popular Rolling Stones HTTP server http://www.stones.com. These servers have very di erent purposes; the former caters primarily to local clients, whereas the latter caters exclusively to remote clients all over the world. In both cases, our analysis showed that remote HTTP accesses were con ned to a very small subset of documents. Using a validated analytical model of server popularity and le access pro les, we show that by disseminating the most popular documents on servers (proxies) closer to the clients, network tra c could be reduced considerably, while server loads are balanced. We argue that this process could be generalized so as to provide for an automated demand-based duplication of documents. We believe that such server-based information dissemination protocols will be more e ective at reducing both network bandwidth and document retrieval times than client-based caching protocols [2]. This work has been partially supported by NSF (grant CCR-9308344).